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(54) Abstract Title 

A collision warning device 

(57) An image capture system captures a scene for display. Image analysis apparatus operates to determine 
the distance from the imaging device to objects of interest within the captured scene and modifies the image 
in response to the determined distance. The described embodiment is for a collision warning device for a 
vehicle wherein the distance between an imaging device and an object within the field of view of the device is 
determined and the image of the scene is modified in dependence with the determined distance prior to 
display for a user in order to give a visual collision warning device. The modification may be a change of 
colour of objects detected as being less than a predetermined distance from the vehicle for example. The 
imaging device may be a multiple region light detector for detecting light received from receiving optics, 
wherein different regions of the light detector can be actuated separately. Control electronics may be used to 
synchronise the timing and direction of illumination of the light source and the actuation of the light detector. 
The time of flight of light signals from the light source to the actuated portion of the detector for all illuminated 
directions may be measured to derive distances. 
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At least one drawing originally filed was informal and the print reproduced here is taken from a later filed formal copy. 
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2374228 

Image Analysis Apparatus 



This invention relates to image analysis apparatus, particularly for collision warning 
and avoidance systems. 

The cost of road traffic accidents, both in terms of economics and human misery is 
vast. For example, in 1999 the US Federal Highway Administration reported 6.3M 
Road Traffic Accidents (RTA) in the USA which left 3.2M people injured and 
41,345 dead. The total economic cost was estimated to be $150Bn. 

Similarly, the Economic Commission for Europe reported that in 1997, European 
RTA injured 6,118,844 people and killed 164,677. The direct costs of Medical 
treatment, emergency services, damage to property & lost economic output were 
estimated to be £36Bn, whilst the total economic cost to the EEC was estimated at 
£150Bn. 

Therefore, much research has focussed on finding ways to avoid collisions and RTA 
by the provision of better driver information and the active and early warning of 
danger and this has lead to a variety of approaches. 

The simplest collision avoidance solutions rely on measuring the distance from the 
vehicle to the nearest vehicle in front and providing a warning light or sound to the 
driver if he is driving too close, given his current speed. 

A simple approach to measuring distance is to use a Laser rangefinder (LRF). These 
devices work on the basis of measuring the time of flight of a laser pulse to a remote 
object and back and calculating the distance from the known velocity of light. A 
limitation of such devices is that they are only able to monitor the distance over a 
pre-defined zone directly in front on the vehicle as shown in Figure 1 (A). The path 
of the vehicle 1 is shown as 2, and the distance monitoring zone is shown as 4. If 
the vehicle is travelling round a bend, the region being monitored will not be 
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focussed on the vehicle's path, but will look either into the path of oncoming traffic 
or into the kerbside as illustrated in Figure 1, situations (B) and (C). 

This is leads to false collision warnings when the road curves or the driver turns 
5 which substantially reduces the benefit of such systems and hence their 
attractiveness to the motorist or commercial driver. In addition, these false warnings 
also make the use of such systems to automatically control vehicle braking, which 
would otherwise improve vehicle reaction time to dangerous circumstances, 
problematic. 

10 

Nonetheless, simple LRF based approaches provide a cost effective solution for 
collision warning or intelligent cruise control systems (where the car velocity is 
automatically controlled to maintain a safe distance behind the car in front) for 
motorway driving. 

15 

To overcome the problem of false warnings on everyday roads and encourage more 
widespread adoption of collision warning, alternative approaches have been tried. 

For example, systems have been developed (such as the Eaton® VORAD® system) 
20 which use forward looking doppler microwave radar techniques to measure the 
distance to a number of local vehicles. Unfortunately such systems are expensive to 
produce because of the sophisticated nature of their components and technology and 
as a result their application has been limited to the commercial vehicle market where 
a higher system price can be tolerated because the vehicles are themselves more 
25 expensive and the economic cost of a collision is higher. 

To realise a lower cost system other workers have been attempting to use a "sensor 
fusion" approach, whereby distance data gathered by a LRF is combined with 
information captured by a video camera and image processing system to try to 
30 eliminate false readings. Such systems often use a priori knowledge about the likely 
size and shape of vehicles and cues from road markings and road furniture to 
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evaluate where the lanes and edges of the road are to check whether the LRF 
distance data is usable and valid. 

However, the road environment is very unstructured from an image processing point 
5 of view and presents a difficult image processing problem requiring substantial and 
expensive computing resources to extract reliable data. Even with such resources, 
these systems find it very difficult to cope with unexpected features in the image; for 
example a child running into the path of the vehicle or some other obstruction in the 
road, because of their reliance on a priori knowledge and this has delayed their 
10 introduction. 

There is therefore a need for image analysis apparatus which can use simple 
technology but which also enables false and true warnings to be easily distinguished 
for each other. 

15 

According to the invention, there is provided an image analysis system comprising: 
an image capture system for capturing an image to be analysed and 

displaying the image to a user; and 

an image analysis apparatus for determining the distance to regions of the 
20 image to be analysed, wherein the image analysis apparatus comprises an output for 

modifying the image displayed by the image capture system in response to the 

determined distance. 

The modification of the visual image enables the user to identify rapidly whether a 
25 warning should be noted or can be ignored. The system preferably further 
comprises a speed sensor, and further comprises means for calculating a safe 
distance based on the output of the speed sensor, the image displayed by the image 
capture system being modified when a distance to a region of the image is less than 
the safe distance. Thus, the system is suitable for use as a vehicle collision warning 
30 system. 

The safe distance may be calculated based additionally on the road conditions. 
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The image capture system may comprise a photodiode array and the image analysis 
apparatus may comprises a laser range finding apparatus, but which may use the 
photodiode array of the image capture system. For this purpose, the photodiode 
5 array is operable in a first mode in which charges are stored on all photodiodes of 
the array in response to light input and then read out in conventional manner to 
capture image data, and a second mode in which the photogenerated signal from a 
selected individual photodiode or group of photodiodes is routed to a time of flight 
measuring circuit to capture surface profile data. 

10 

The image analysis apparatus may comprise a maximal length sequence generator 
for generating a modulation signal, and a cross-correlator for obtaining the time 
delay of the time delayed reflected nodulation from a comparison of the 
modulation signal and the time delayed reflected nodulation signal. 

15 

Means for determining the trajectory of a body (i.e. a vehicle) carrying the image 
analysis system may be provided. The output for modifying the image displayed by 
the image capture system may then be created only for regions of the image to be 
analysed lying within the trajectory of the body. 

20 

The invention also provides a method of providing a collision warning to the driver 
of a vehicle, comprising: 

obtaining an image of the scene in front of the vehicle; 

obtaining range information for objects in front of the vehicle; 
25 modifying the image of the scene for objects within the scene having a range 

less than a determined level. 

The modification may comprise changing the colour of the image in areas 
corresponding to the objects. 

30 

Examples of the invention will now be described in detail with reference to the 
accompanying drawings, in which; 
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Figure 1 is used to illustrate some problems with collision avoidance 
systems; 

Figure 2 shows a first example of collision avoidance system of the 
invention; 

5 Figure 3 shows a preferred image capture and range finding apparatus for 

use in the system of the invention; 

Figure 4 shows the photodiode array of Figure 3 in more detail; 
Figure 5 shows a second embodiment of system of the invention; 
Figure 6 shows a third embodiment of system of the invention; 
10 Figure 7 shows how trajectory information can be used; and 

Figure 8 shows a fourth embodiment of system of the invention. 

The simplest version of the proposed system is illustrated in Figure 2. 

15 A video camera 10 and a real-time 3D surface profile measurement system 12 are 
arranged so that their fields of view 14, 16 are co-incident and aligned with the road 
ahead. By surface profile measurement system is meant a system which measures 
distance to different points within a field of view. The video camera has a video 
output providing an intensity signal I(X,Y) for each point in the field of view and 

20 the profile measurement system has an output indicating the range R(X,Y) to each 
point in the filed of view. Control circuitry 18 generates sync pulses to ensure that 
the scanning of the video camera 10 and 3D surface profile measurement system 12 
are synchronised; so that the same point in the field of view is viewed by each 
system at the same time. 

25 

A vehicle velocity sensor 20 measures the vehicle velocity and the safe distance 
(Rmin) is computed in processor 22 taking into account the vehicle brake 
performance and driver reaction time. A comparator 24 compares the measured 
distance at each scanned point in the field of view R(X,Y) with the safe distance 
30 Rmin and generates a signal whenever the measured distance to a point within the 
field of view is less then the safe distance (R(X,Y)<Rmin). This signal may be 
used to trigger an audible warning provided by audio output 26. 
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In addition, the warning signal operates an overlay switch 28 which replaces the 
current intensity value I(X,Y) of the pixel being readout from the video camera with 
an alternative preset warning colour generated by a separate source 30; e.g. red. In 
5 this way, those areas or objects within the field of view of the video camera which 
are dangerously close are highlighted in the video signal. 

The video signal is then displayed to the driver through a conventional video screen 
or head up display 32, and the highlighting indicates to the driver which objects or 
10 vehicles are too close. 

Figure 2 shows separate LRF and video imaging systems 10, 12. It is instead 
possible to simplify the design of the video camera and the surface profile 
measurement system, in particular by sharing the light receiving hardware. A 
15 combined video and profile measurement system will now be described with 
reference to Figure 3. 

A sequentially pulsed laser beam output from a laser 40 is scanned across the field 
of view 41. The scanning is achieved either by scanning the laser itself, or 
20 preferably using a scanned mirror 42 implemented using known techniques such as 
galvanometer or piezo-electric drives 44. 

A stationary, receiving optical system 46 is arranged to collect all the light from the 
remote object and focus it onto a photodiode array 48. The photodiode array 48 is 
25 connected to a pre-amplifier, pulse discriminator 54 and timing electronics 56. 

Control electronics 52 control the scanning of the laser beam in azimuth and 
elevation (X, Y) and the timing of laser pulsing. Each laser pulse is reflected from 
objects in the field of view 41, collected by receiving optics 46 and focused onto the 
30 photodiode array 48 to generate an electrical pulse, in a part of the array where the 
part of the object illuminated by the laser spot is focused. 
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The control electronics apply logic level signals to the relevant X and Y control 
lines of the X-Y addressed array so that the photodiode illuminated by the image of 
the laser spot is connected to a pre-amplifier and time of flight detection electronics 
56. The reflected laser pulse is captured by this photodiode and the resultant 
5 electrical signal routed to the electrical pulse detector and time of flight (TOF) 
measurement circuitry 56. This computes the TOF of the laser pulse to the spot on 
the remote object and back to the photodiode on the X-Y addressed array and hence 
distance from the remote object to the X-Y addressed array. 

10 This process is repeated for many points within the field of view to measure the 
range of objects within the field of view. If the image formed of the laser spot is 
larger than a single pixel then the control electronics can cause the detector to 
address a group of adjacent photodiodes (e.g. a 2 x 2 sub-array of photodiodes) in 
parallel to optimise collection and detection of the laser energy. 

15 

Because the control electronics 52 is controlling the laser scanning and laser pulse 
timing, it is able to build up a matrix of numbers comprising the laser scan azimuth 
and elevation (X,Y) and the distance (Z) to the remote object at that laser line of 
sight which represents the 3D surface profile of the remote object. 

20 

It can be seen that with this approach, the only moving part of the system is a 
scanned mirror which only need be sufficiently large to steer the laser beam. This 
avoids the high cost of a precision motorised pan and tilt head and enables a high 
scan rate. Furthermore, because the laser and receiving optical paths can be kept 
25 completely separate there is no risk of optical crosstalk. 

To minimise size and cost, the laser scanning system 44 may be implemented in a 
number of ways including using electro-magnetically or piezo-electrically scanned 
mirrors or by mounting a laser chip on a micro- machined silicon or compact piezo 
30 electric structure. 
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The performance of the system can be substantially improved by replacing the 
pulsed laser source with a modulated laser source and the pulse discriminator by a 
cross-correlation system. Such systems are known, for example, from DE19949803 
to Denso Corp. In particular, the system may include a signal source such as a laser 
5 for supplying a modulation signal and a transmission system connected to the signal 
source for transmitting a transmitted optical signal modulated by the modulation 
signal. 

The modulation signal may be, for example a maximal length sequence. In this 
10 way, for a given laser peak power, greater energy can then be delivered to the 
remote object which improves the signal to noise ratio and hence maximum range of 
the system. A reception system is then be provided for receiving a reflected and 
delayed version of the transmitted signal, and a cross-correlator for obtaining the 
time delay. The cross correlator can be arranged to determine, at a coarse 
15 resolution, the time delay of the modulation signal needed to maximise the 
correlation between the time delayed modulation signal and the received signal. The 
cross correlator can then determine, at a finer resolution than the coarse resolution, 
the correlation between the modulation signal and the received signal as a function 
of the time delay of the modulation signal with respect to the received signal in a 
20 smaller time delay range around the determined time delay. A measure of distance 
is calculated from the time delay of the modulation signal needed to maximise the 
correlation between the time delayed modulation signal and the received signal. 

The cross-correlator can be implemented digitally and the sampling frequency of the 
25 cross-correlator set to be a multiple of the maximal length sequence generator clock 
frequency. This oversampling approach enables the distance resolution of the 
system to be improved; and the efficient signal processing method using coarse and 
fine cross-correlators minimises the processing power needed. 

30 This oversampling approach also increases the system's immunity to interference 
from other like systems being operated by nearby vehicles (e.g. in adjacent lanes). 
This is because the correlation peak detected in a maximal length sequence (MLS) 
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based TOF system using a specific oversampling factor is insensitive to another 
MLS signal generated with a different oversampling factor. For example, there is 
little correlation between a MLS of oversampling factor 5 and a of oversampling 
factor 6, even if the MLS signals are of the same order. 

5 

This property can be used to advantage in a collision avoidance context as follows: 

a) if a collision avoidance system detects the presence of a reflected signal from 
an adjacent system using the same oversampling factor, it can switch to a different 
oversampling factor and/or MLS order to avoid interference between the two 

10 systems; 

b) alternatively, the system can sequentially cycle through a number of different 
oversampling factors and MLS orders over time so that the probability of 
interference between two adjacent systems is much reduced, 

15 

For example, the oversampling factor and MLS order could be calculated by a 
pseudo random number generator which is seeded with a number based on the time 
the car is started so that the probability of two cars running on the same 
oversampling factor at the same time would be very small. 

20 

The preferred MLS technique outlined above will now be described in greater 
detail. An MLS generator generates an MLS signal. The MLS generator clock 
signal is derived from the system master clock by a divider so that the MLS clock 
frequency is a known sub-multiple M of the master clock signal. In effect, the MLS 
25 is stretched in time by factor M. The "stretched" MLS signal causes the laser to 
emit an optical stretched MLS signal, the returned signal from the objects in the 
field of view being digitised and passed to coarse and fine cross-correlation 
calculation units. 

30 The coarse cross-correlation unit is clocked at the MLS clock frequency and hence 
correlates a sub-sampled version of the digitised reflected MLS signal and original 
stretched MLS transmitted signal. The output from this cross correlation unit is a 
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peak which is detected and which indicates the coarse time delay of the reflected 
signal. 

The control electronics then causes the fine cross-correlator to calculate the cross- 
5 correlation of the transmitted and reflected signals only in the region of the 
calculated coarse time delay . Typically, the fine cross-correlation function would 
be calculated for 2M samples before and after the coarse time delay. The output of 
the fine cross correlator is the cross correlation function of the transmitted and 
reflected signals in the region of the peak. 

10 

The shape of the correlation peak for a PRBS signal such as an MLS is a triangular 
pulse. The cross-correlation operation may be viewed as being similar to 
convolving the MLS with a delayed version of itself and then sampling the result at 
a frequency equal to the cross correlator clock frequency. Therefore, the shape of 
15 the correlation peak output by the cross-correlation unit is given by the convolution 
function of two identical pulses of width T, which is a triangular pulse sampled by 
the cross correlator clock frequency. 

The photodiode array of the LRF system may be used as the image acquisition 
20 system for the video camera. A simplified schematic of one possible design of a 
photodiode array for this purpose is shown in Figure 4. A 2 X 2 array is shown for 
simplicity, whereas a much larger array will be used in practice. The device 
consists of an array of photodiode pixels 60, each of which comprises a photodiode 
(PD11 to PD22) and associated transistor (TR11 to TR22), which are configured 
25 and drive to act as analogue switches. 

For standard video imaging applications, the device is operated in an integration 
mode where incident illumination is focussed upon its surface. The incident 
illumination generates charge within each photodiode by the photoelectric effect. 
30 During this integration period, connections XI, X2, Yl and Y2 are all held low so 
that all transistors are off and the photodiodes are electrically isolated. The photo- 
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generated charge then accumulates in each photodiode and is stored on the self- 
capacitance of the photodiode. 

Once sufficient photocharge has been collected, the device is readout as follows. 
Input XI is taken to a high potential so that TR1 is turned on thereby allowing 
charge to flow between the column and a charge sensitive amplifier 62. Then input 
Yl is pulsed high for addressing a row of pixels, turning TR11 on and allowing the 
photo-generated charge stored on photodiode PD11 to flow through TR11 and TR1 
to the output amplifier 62 where the charge is converted to a voltage. This creates 
an output signal whose amplitude is proportional to the level of charge stored on 
PD1 1 and hence the level of light incident on TR1 1 . 

After the self capacitance of PD11 has been discharged, input Yl is taken low and 
input Y2 is taken high, allowing the stored charge on PD12 to be readout. In this 
way, a column of pixels is read out in turn. 

After all the charge collected by PD12 has been discharged, Y2 is taken low and X2 
is taken high to allow PD21 and PD22 (the pixels in the next column) to be readout 
sequentially by pulsing Yl and Y2 in the manner described above. 

It can be seen that this process allows the 2 X 2 array to be scanned and an electrical 
signal that is the analogue of the incident illumination generated. In normal 
operation, larger numbers of photodiode are used, e.g. 512 x 512, to increase 
resolution. The readout sequence and sensor scanning can be arranged to generate 
a standard video signal. 

In addition, it may be noted that the basic structure described here has been 
simplified for the purpose of describing the proposed invention. Practical X-Y 
addressed photodiode arrays are generally fabricated as single complementary metal 
oxide semiconductor (CMOS) large scale integrated circuits (LSI) which include 
many refinements such as on-chip clock circuitry to generate the pulse sequences for 
electrodes XI to Xn and Yl to Yn on-chip and additional pixel and/or column level 
circuitry improve amplification and detection of the photo-charge. 
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For 3D profile measurement, the X-Y addressed photo-diode array can be utilised 
not in an integrating mode, but as a multiplexer, whereby only the individual 
photodiode receiving the reflected image of the laser spot on the remote object is 
5 addressed, as discussed above. When only one or a small number of the 
photodiodes is connected to the receiving amplifier and time of flight electronics at 
any one time, the background offset signal will be limited to that generated by the 
part of the field of view focussed onto the individual photodiode/photodiode group, 
rather than from the whole of the field of view of the optics. 

10 

In a preferred embodiment, the X-Y sensor, time of flight measurement system and 
control electronics are fabricated on a single integrated circuit to minimise 
manufacturing cost. The photodiodes can be manufactured and operated as 
avalanche photodiodes to provide signal amplification by the avalanche effect, prior 
15 to signal detection. 

The laser scanning pattern will often be a repeating pattern arranged to cover the 
optical field of view whilst providing adequate time resolution to measure the 
position of moving objects in the field of view. The pattern is typically arranged as a 
20 conventional raster scan for ease of display on conventional monitors. However, it 
can be seen that other patterns may be used. One useful pattern is a spiral scan 
pattern where by controlling the velocity of the laser scan, increased spatial 
resolution may be achieved in the centre of the scan whilst still maintaining a low 
spatial resolution to detect objects appearing at the periphery of the scan. 

25 

In order to operate the array in a normal imaging mode, rather than a multiplexed 
time of flight detector mode, the sequence of pulses applied to the X-Y addressed 
array is returned to a conventional video scanning sequence. In order to enable the 
two systems to operate continuously, a combined scanning sequence will be applied 
30 which can, for example, provide a range finding operation and an imaging operation 
in alternate frames. By toggling the system between capturing a 3D scan and a 
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conventional image scan both a video image sequence and 3D sequence can be 
captured and overlaid on one another. 

The use of the photodiode array in two different modes enables the operation to be 
optimised for each mode. For the LRF operation, it is important that only the 
photodiode or local group of photodiodes receiving the image of the laser spot at 
any point in time are addressed; i.e. that the laser scanning and the photodiode array 
scanning are synchronised. This would normally require extremely precise 
calibration of the scanner and optical system. However, if the laser is scanned whilst 
the sensor is in an imaging mode, an image of the laser path can be collected by the 
control electronics. This image can be used to determine the precise path of the laser 
beam image on the surface of the photodiode array and hence set up the correct 
addressing sequence for the X-Y addressed array and/or laser pulsing sequence, to 
ensure synchronisation in the multiplexed time of flight detector mode. Thus, the 
normal addressing mode is used as a calibration stage for the higher performance 
multiplexing mode. In effect, the system can be self calibrating which is a major 
benefit for systems which have to operate over large temperature ranges. 

Because the same detector is used to capture a 3D surface profile and standard 
image, the registration between the standard image and 3D data is near perfect. 

To those skilled in the art, it can be seen that there are a variety of ways of 
interleaving video image capture and 3D surface profile measurement. For example, 
the clocking of the photodiode array could be arranged such that alternate pixels, 
groups of pixels or lines could be used for image capture and surface profile range 
capture. With these alternative approaches, a mosaic or stripes of infra read 
bandpass filters can be applied to the surface of the photodiode array aligned with 
those pixels or lines which are to be used for range measurement to improve 
discrimination of the laser illumination against background illumination and hence to 
increase range. 
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Various rules may be applied to determine when the video image should be modified 
to provide a warning signal. In a preferred embodiment, vehicle velocity is used to 
calculate a number of different distance limits representing different safety margins. 
Additional comparators and overlay switches are provided to colour code the video 
5 signal depending upon the category. For example, Red could be used to indicate that 
the object is so close as to be dangerous and orange could be used to indicate that 
additional care needs to be taken. Flashing of parts of the display may also be used 
as a warning. 

10 Hence, by colour coding the driver display, the driver is able to identify the risk 
posed by different objects and, because the driver is able to correlate the collision 
warning with an object in the field of view, is able to determine whether it is a false 
warning or not. This overcomes one of the major disadvantages of the simple LRF 
systems described above, where the driver has no information about what has 

15 caused the collision warning. 

The system may be improved further by providing additional sensors to monitor 
other important factors such as the road temperature, whether it is raining and road 
surface moisture level. These factors would be used to refine the safe distance to 
20 take into account road and environmental conditions and the modify the colour 
coding of the warning display accordingly. 

Preferably the 3D surface profile measurement system operates at near infra red 
wavelengths to maximise penetration and range, even in the presence of rain, snow 
25 or fog. 

It can also be seen that this system could be deployed not only to the front of the 
vehicle, but also to the side and rear of the vehicle to provide all round warnings to 
the driver. If used to the sides of the vehicle, the collision warning system could be 
30 employed as an active side view "mirror", where objects which pose a threat if the 
driver is to change lane are highlighted. A further advantage is that the camera/3D 
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surface profile measurement system can be positioned to eliminate the usual door 
mirror blind spot. 

Figure 5 shows a system which can use the combined imaging and range finding 
system described above. The same reference numerals are used as in Figure 2 for 
the same components. 

A synchronisation control circuit 70 switches the combined sensor 72 (with a single 
field of view 73) sequentially between imaging and 3D surface measurement modes. 
During the imaging mode, the image data is readout in the normal manner and 
stored in a frame buffer 74; i.e. the image data is stored in a block of memory. 

During the 3D scanning mode, the stored image data is readout from the frame 
buffer 74 in synchronism with the 3D surface profile data. The surface profile data 
is compared with the computed safe distance criteria from the processor 22, which 
receives vehicle velocity and road condition data from sensors 76. There may be a 
number of levels, for example ranges defined as critical, preferred or safe, as shown 
in Figure 5. The stored image data is colour coded for display in the manner 
described above, using colours from the source 30 and an overlay unit 28. 

An alternative but equally effective implementation would be to store the 3D data in 
a frame buffer and read the 3D data out in synchronism with the image data. In 
either case, the range measurement and visual image acquisition are carried out in 
sequence, and one of the sets of data is stored, so that the two sets of data can be 
combined in order to drive the display. 

In a preferred embodiment, all of the collision warning system circuitry would be 
integrated into a large scale integrated circuit which also includes the photodiode 
array and associated circuitry used for the imaging/surface profile measurement 
sensor system. This is to minimise manufacturing cost. 
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In summary, the simple collision warning systems described above will colour code 
all objects within the field of view which are too close to the vehicle so that the 
driver can exercise judgement over which objects are threats and take the necessary 
avoidance action, which is a significant improvement over existing systems, which 
5 provide a warning of potential collision, but do not enable the driver to understand 
where the potential threat comes from. 

However, some objects will be inevitably be coded as being too close whilst not 
posing threat of collision. Examples include objects on the pavement or on the other 
10 side of the road. To overcome this issue, the basic system can be improved as 
described below. 

The basic systems described above can be improved by modifying them as shown in 
Figure 6. Again, the same reference numerals are used as in Figures 2 or 5 for the 
15 same components. The system of Figure 6 differs in the way the safe range is 
calculated. 

A gyroscope 80 or other sensor capable of detecting the turning motion of the 
vehicle is used (for example a steering column encoder). 

20 

The turning vector, and vehicle velocity vector from sensor 76 and knowledge of 
the vehicle size are used to calculate the vehicle trajectory and hence the outside 
perimeter of the projected swept volume of the vehicle using standard equations of 
motion. This is carried out by vehicle trajectory calculation unit 82, The velocity 
25 sensor 76 and road and environment sensors 77 are used to compute a safe distance 
limit in the normal manner in the processor 22. 

The safe distance limit from the processor 22 is then combined with the projected 
swept volume from unit 82 to define an unsafe volume zone as illustrated 
30 schematically in two dimensions in Figure 7. Figure 7 shows the vehicle trajectory 
90 of the vehicle 91 which defines a volume zone 92. A safe distance limit 94 
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divides the volume 92 into a region 96 close to the vehicle and a region 98 at a safe 
distance. 

Typically, the unsafe volume zone is computed in a cartesian co-ordinate system 
5 with the front of the vehicle (or the vehicle centre of mass) positioned at the origin. 
The unsafe volume zone is calculated by unit 82. However, the 3D surface profile 
measured by the surface profile and image system is a matrix of numbers which 
represent the distance to points in the field of view (the "real world") along lines 
projected from the each pixel on the surface of the photodiode sensor array via the 

10 perspective centre of the lens. As the geometry of the optical system of the 3D 
surface profile measurement system is known, then using the standard mathematical 
rules of central perspective a co-ordinate transformation function can be computed 
in unit 84 to transform co-ordinates in the cartesian system used to compute the safe 
volume zone to those in field of view of the scanning system. This enables a 

15 volume map in the coordinates of the scanning system to be built up in unit 86. 

With adequate control over the optical system tolerancing, this function need only 
be computed or measured once using standard photogrammetric calibration 
techniques and then pre-programmed into each system. 

20 

During operation, the co-ordinate transformation function is used to transform the 
Cartesian co-ordinates of the perimeter of the safe volume zone to the "real world" 
co-ordinate system measured by the 3D surface profile measurement system. 

25 The comparator 24 can then compare the measured 3D surface profile as it is read 
out with the unsafe volume zone and determine whether any object in the field of 
view falls within the perimeter of the unsafe zone. If it does an audible warning may 
be sounded and the potentially dangerous part of the field of view highlighted on the 
video image in the same manner as described above. 

30 

With this improved system, only those parts of the 3D surface profile data that fall 
within the projected unsafe volume 96 are considered, and hence only those objects 
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that pose a threat of collision will be highlighted, substantially reducing the 
possibility of false warnings. 

This approach has a further benefit in that because only the perimeter of the unsafe 
5 zone needs to be computed and transformed into the 3D surface profile co-ordinate 
system, rather than the whole image region, computation time is minimised. 
However, those skilled in the art will realise that alternative implementations are 
possible based on the basic principles outlined above. For example, the 3D surface 
profile could be transformed into the cartesian co-ordinate system of the vehicle 
10 and the comparison of the surface profile and unsafe volume zone made in cartesian 
co-ordinates. 

A preferred embodiment for a lower cost implementation is to apply the approach 
described above to a 2D surface profile captured from a horizontal plane which is 
15 parallel to the road surface at a height chosen to intersect other vehicles and 
potential collision threats, while not intersecting the road. 

Whilst this does not give height data, which may be important in some applications, 
it does simplify the computational requirements and as the 2D surface profile can be 
20 gathered by taking the surface profile data from a single row (or a group of adjacent 
rows) of photodiodes in the surface profile/imager system, the optics of the 
pulsed/modulated illumination can be simplified so that the illumination is focussed 
into that horizontal plane, rather than over the whole field of view. These factors 
simplify the overall system implementation and reduce cost. 

25 

Another implementation uses the unsafe zone projection to control the operation of 
the 3D surface profile measurement system so that the 3D profile measurement is 
limited to the part of the field of view within the projected unsafe zone. This 
approach may be used to speed up the 3D surface profile measurement process, by 
30 reducing the number of points to be measured or may be used to extend the 
measurement time per measured point to improve signal to noise ratio and hence 
maximum range. 

18 



A number of unsafe volume maps can be generated, representing different degrees 
of risk and multiple comparators used to colour code the video image by the level of 
risk. 

In the systems described above, the projected path of the vehicle is computed using 
information from a gyroscope or steering wheel sensors and velocity sensors. This 
is used to compute the anticipated swept volume of the vehicle path which is 
transformed into the co-ordinate system of the measured surface profile. 

Figure 8 shows a modification in which instead of a simple comparison operation, 
the surface of intersection of the measured real world surface profile and anticipated 
swept volume of the vehicle path is computed, so that the distance from the vehicle 
to the nearest object along the projected vehicle trajectory can be extracted from this 
intersection surface. This parameter can then be output from the system and used by 
automatic systems to control the vehicle velocity so as to avoid a collision or 
minimise the impact. In other words, this modification enables the proposed system 
to be used as a collision avoidance sensor. 

The system of Figure 8 again comprises a combined sensor controlled with 
sychronisation control 70. The buffer 74 stores the 3d profile (i.e. range) data, and 
is used to calculate whether any object in the field of view intercepts with the 
projected volume of the vehicle. This is determined in unit 100, which in turn 
enables the distance to the nearest point within the projected volume to be 
calculated, in unit 102. An output 104 is provided which indicates the distance to 
the nearest object in the projected vehicle volume. 

The projected vehicle volume is obtained in the same manner as described above, 
namely using a gyroscope 80, speed sensor 76, trajectory calculation unit and co- 
ordinate transformation unit 84. 
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One simple approach to computing the distance to the nearest object would be to 
sequentially step a projection of a surface equivalent to the frontal area of the 
vehicle along the anticipated vehicle trajectory until an intersection with the 
measured surface profile is found. Another approach would be to carry out a binary 
search, whereby the projection of the frontal area of the vehicle is set at a point 40m 
along the anticipated vehicle path, if the projection is behind the measured surface 
then the projection of the frontal area of the vehicle is set at 20m and so on, until 
the distance to the nearest point of intersection is known. 

It can be seen that a dual purpose system can be constructed by combining Figure 8 
and Figure 6 which would provide rapid feedback of the distance of the nearest 
object in the path of the vehicle for automatic vehicle control with a collision 
warning to the driver. 
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Claims 



1 . An image analysis system comprising: 

an image capture system for capturing an image to be analysed and 
displaying the image to a user; and 

an image analysis apparatus for determining the distance to regions of the 
image to be analysed, wherein the image analysis apparatus comprises an output for 
modifying the image displayed by the image capture system in response to the 
determined distance. 

2. A system as claimed in claim 1, further comprising a speed sensor, and 
further comprising means for calculating a safe distance based on the output of the 
speed sensor, the image displayed by the image capture system being modified when 
a distance to a region of the image is less than the safe distance. 

3. A system as claimed in claim 2, further comprising a road condition sensor, 
and wherein the safe distance is calculated based additionally on the road condition 
sensor output. 

4. A system as claimed in any preceding claim, wherein the output for 
modifying the image displayed by the image capture system changes the colour in 
parts of the image displayed corresponding to selected regions of the image to be 
analysed. 

5. A system as claimed in any preceding claim, wherein the image capture 
system comprises a photodiode array. 

6. A system as claimed in claim 5, wherein the image analysis apparatus 
comprises a laser range finding apparatus. 

7. A system as claimed in claim 6, wherein the image analysis apparatus 
comprises a maximal length sequence generator for generating a modulation signal, 
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and a cross-correlator for obtaining the time delay of a time delayed reflected 
nodulation signal from a comparison of the modulation signal and the time 
delayed reflected modulation signal. 

5 8. A system as claimed in claim 7, wherein the cross correlator is arranged to 
carry out the steps of: 

determining, at a coarse resolution, the time delay of the modulation signal 
needed to maximise the correlation between the time delayed modulation signal 
and the modulation signal, 
10 determining at a finer resolution than the coarse resolution, the correlation 

between the time delayed modulation signal and modulation signal as a function of 
the time delay of the time delayed modulation signal with respect to the 
modulation signal in a time delay range around the determined time delay, and 
outputting a measure of distance calculated from the time delay of the 
1 5 modulation signal needed to maximise the correlation between the time delayed 
modulation signal and the modulation signal. 

9. A system as claimed in claim 7 or 8, wherein the cross-correlator 
comprises: 

20 a coarse cross-correlator for coarsely determining the time delay of the 

modulation signal needed to maximise the correlation between the time delayed 

modulation signal and the modulation signal, and 

a fine cross-correlator for calculating the correlation between the time 

delayed modulation signal and the modulation signal as a function of the time 
25 delay of the modulation signal with respect to the received signal in a time delay 

range around the time shift determined by the coarse cross-correlator. 

10. A system as claimed in claim 9, wherein the coarse cross correlator is 
clocked at a first frequency and the fine cross-correlator is clocked at a higher 

30 second frequency. 



22 



11. A system as claimed in any one of claims 6 to 10, wherein the laser range 
finding apparatus uses the photodiode array of the image capture system. 

12. A system as claimed in claim 1 1 , wherein the photodiode array is operable in 
a first mode in which charges are stored on all photodiodes of the array in response 
to light input and then read out in conventional manner to capture image data, and a 
second mode in which the photogenerated signal from a selected individual 
photodiode or group of photodiodes is routed to a time of flight measuring circuit to 
capture surface profile data. 

13. A system as claimed in any preceding claim, further comprising means for 
determining the trajectory of a body carrying the image analysis system. 

14. A system as claimed in claim 13, wherein the output for modifying the 
image displayed by the image capture system is created only for regions of the 
image to be analysed lying within the trajectory of the body. 

15. A system as claimed in claim 13 or 14, further comprising means for 
determining the distance to the nearest object within the trajectory. 

16. A system as claimed in any preceding claim comprising a vehicle collision 
warning system. 

17. A method of providing a collision warning to the driver of a vehicle, 
comprising: 

obtaining an image of the scene in front of the vehicle; 
obtaining range information for objects in front of the vehicle; 
modifying the image of the scene for objects within the scene having a range 
less than a determined level. 

18. A method as claimed in claim 17, wherein the modification comprises 
changing the colour of the image in areas corresponding to the objects. 
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19. A method as claimed in claim 17 or 18, wherein the determined level takes 
into account the vehicle speed. 

5 20. A method as claimed in any one of claims 17 to 19, wherein the image is 
modified only for objects within a trajectory of the vehicle, wherein the trajectory is 
obtained from speed and direction information. 
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